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Notes 

MODULE : EVALUATE REGULAR AND COOPERATIVE 
VOCATIONAL PROGRAMS 

Competencies/Tasks: 

In an actual vocational education situation, regular or cooperative education, you will 
evaluate your program or an activity within your program. 



Objectives: 

The student will be able to: 

1 . Identify the various methods that can be used to evaluate a regular or 
cooperative education program. 

2. Explain, verbally or in writing, why the evaluation of a program is necessary. 

3. Explain, verbally or in writing, what should be evaluated in a program. 

4. Explain, verbally or in writing, what techniques can be used for the evaluation. 

5. Explain, verbally or in writing, who should be involved in the evaluation 
process. 



Overview of Module: 

Suggestions for the teacher educator. As the students complete each unit, have them 
relate it to theirown program evaluation situation. Example: In Unit II, have the students 
determine what activity should be evaluated in their own program. 

It is a little much to expect a complete program evaluation to be completed during the 
short time it takes to cover this module. However, (he result of the module could be the 
subject of an independent study in vocational education. Time schedule - three to five 
hours. 



Suggested Resources: 

1. Media: 

A. Videotape - "What is Cooperative Vocational Education?" Source: The 
Media Center, State Fair Community College 

2. Assignments: 

A. You are required to read the information contained in this module. 

B. You may wish to read one or more of the following references: Lawson 
and Wentling, Evaluating Occupational Education and Training Programs; 
Tuchman, Evaluating Instructional Programs; and/or Darcy, Vocational 
Education Outcomes: Perspectives for Evaluation. 

C. You will demonstrate your knowledge of the concepts and procedures 
involved in the evaluation of a regular or cooperative vocational education 
program by completing an evaluation. 
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D. You will demonstrate your competency by developing an evaluation plan, 
following through with an actual evaluation, analyzing and interpreting the 
data collected, and preparing a final report of conclusions and recomm 
endations. 

3. References: 

Ash, L.C., Instruments and Procedures for the Evaluation of Vocational/ 
Technical Education, Washington, DC, American Vocational Association, 
1972. 

Darcy, Robert L, Vocational Education Outcomes: Perspectives for 
Evaluation, R&D Series No 163, Columbus, OH, The National Center for 
Vocational Education, Ohio State University, 1979. 

Erickson, R.C. and T. L. Wentling, Measuring Student Growth: Techniques 
and Procedures for Occupational Education, Boston, MA, Allyn & Bacon, Inc., 
1976. 

Lawson, Tom E. and T. L. Wentling, Evaluating Occupational Education and 
Training Programs, Boston, MA, Allyn & Bacon, Inc., 1975. 

Norton, R.E. and Norval McCaslin, Vocational Education for Special Groups, 
Edited by James E. Wall, Washington, DC, American Vocational Association. 

Tuckman, Bruce W., Evaluating Instructional Programs, Boston, MA, Allyn &' 
Bacon, Inc., 1985. 

National Center for Research in Vocational Education, Leadership and 
Training Series: Develop Local Plans for Vocational Education, Part II, 
Columbus, OH, National Center for Research in Vocational Education, Ohio 
State University. 

The National Center for Research in Vocational Education Leadership and 
Training Series, Direct Program Evaluation, Columbus, OH, The National 
Center for Research in Vocational Education, Ohio State University. 



Content/Jnstructional Strategies: 
Prerequisite information: 

Before undertaking this module students should have mastered: 

1 . Developing program goals and objectives 

2. Conducting surveys and follow-up studies 

3. Designing and/or selecting program evaluation instruments. If you do not 
already have these competencies, you may wan to complete the information 
and practice activities in the following modules: 

Develop Program Goals and Objectives AAVI U Module A-6 
Conduct a Student Follow-up Study AAV'M Module A- 10 
Evaluate your Vocational Program AAVIM Module A-1 1 
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Introduction: 

This module provides the basic information necessary to conduct or supervise regular 
and cooperative vocational education program evaluations. 

Vocational education is a dynamic process; fairly dynamic shifts in emphasis become 
apparent from decade to decade. Evaluation must also be viewed as a dynamic 
activity. Objectives must be reviewed and updated continuously as well as the devices 
and procedures that measure them. Criteria must be re-evaluated over time as more 
data is accumulated. Long term evaluations do help provide perspective but should not 
be expected to span periods over five years to remain timely and meaningful. 

Designing an evaluation system for regular and cooperative vocational education 
programs: 

#1 
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A. Why evaluate vocational education programs? The word "evaluation" has a 
threatening effect on most educators. It brings forth thoughts of tenure, 
salary, promotion, facilities and accreditation. Program evaluation has as its 
function to find out if we are fulfilling our objectives, if the results are meeting 
the needs and to provide evidence on which to make improvements in the 
program. There are several reasons why evaluation is important: 

1. As a planning aid. Information collected during an evaluation can provide 
measures of resources, limitations and possibilities essential to 
establishing and assessing objectives plus developing plans. It can help 
determine the worth of an existing program plan if the overall plan is 
reasonable and if goals have been achieved. 

2. As a decision making aid. Decisions, no matter what their importance, 
should be based on valid and reliable information. Rational decisions that 
are defensible and justifiable to staff, administration and the public can be 
based upon evaluation findings. 

3. As a program personnel aid. A wel! designed evaluation system will 
identify deficiencies and strengths of program personnel and therefore 
help them to improve their performance. 

4. As a student program aid. Since evaluation determines the value of 
programs and offers suggestions tor improvement, it will ultimately affect 
the students. The improvement of programs is the most important result 
of an evaluation. When deficiencies within a program are defined, 
corrective action can be taken. 

5. As a financial accountability aid. The public and government require 
evidence to determine whether the outcomes of a program are worth the 
investment. This requires more than a financial audit of accounting 
records. It requires a presentation of the program results, such as the 
placement of qualified students in jobs, in relation to costs and objectives. 
An evaluation system can help meet the financial accountability 
requirement. 

B. What should be evaluated? The primary role of program evaluation is to provide 
relevant information to key decision makers so that they can make informed 
judgements. Therefore, the evaluation a ;tivity should start with determining what 
decisions are to be made. The types of decisions made by educators may be 
classified as: 

1. Planning. Decisions which are concerned with needed improvements, 
major goals, and specific objectives. 



2. Programming. Decisions which specify procedures, personnel, facilities, 
resources and time requirements for planned programs. 
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3. Implementing. Decisions which direct program activities. 

4. Restructuring. Decisions necessary for terminating, sustaining, adjusting or 
modifying the program or its parts. 

The components of a program which contribute to its success are: 
administration, personnel, objectives, evaluation system, content, 
students, resources, guidance and counseling, placement and other 
ancillary services. The degree which these components are considered 
for evaluation will vary. The evaluation of an institution or depart ment 
may be more concerned with the administrative, instructional, guidance, 
placement, counseling, goals of the organization, etc. If the evaluation 
concerns an individual program or activity, the focus of the components 
will be more specific. 

When evaluating a course, the areas of concern are: students served, 
goals and objectives, organizations, personnel, content, teaching 
methods, learning assessment and supplies, equipment and facilities. If 
the evaluation is concerned with a segment of a course the scope would 
be more limited. 

C. What techniques should be used in evaluation? The design of the evaluation 
system should be directed towards the needs of the local vocational education 
organization. No design can be universal for all situations. However, there are 
various techniques that can be combined to build an evaluation system. 
These techniques, with a brief description, include: 

1 . Student assessment. The measurement of student performance is the 
most widely used technique to evaluate or form a basis for program 
evaluation. 

2. Graduate assessment (follow-up study). Follow-up studies of graduates 
can provide data from or about individuals after they have had similiar or 
comparable experiences. A follow up assessment involves contacting 
each graduate, usually by a mailed questionnaire, telephone interview, or 
personal interview. This contact can provide opinions, ratings and 
suggestions by the graduate based on his/her perception of the programs 
strengths and weaknesses. Most graduates are capable and willing to 
evaluate their job preparation and suggest methods for program 
improvememnt. The graduate assessment can provide evaluation 
information on the total institution, a vocational program, individual 
programs, activities or courses. 

3. Employer assessment (follow-up study). Employer assessment of 
graduates and/or cooperative education students can provide 
performance ratings and suggestions for improving the vocational 
program. The employer assessment may be conducted concurrently with 
the graduate assessment or independently. It can be conducted using 
a mailed questionnaire, telephone interview, or personal interview. The 
information gathered should answer pre-specified questions that were 
formulated by the evaluator and/or staff. The information obtained 
concerning the same students through the utilization of student 
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assessment, graduate assessment and employer assessment can be 
integrated to help determine the appropriateness of program and course 
objectives and the competencies required by the employers. The results 
would enhance identifying program and course strengths and 
weaknesses. This activity can focus on the total institution, a vocational 
service area, an individual program, a course or an activity. 



EVALUATION OF EMPLOYEE'S 
PREPARATION FOR EMPLOYMENT 
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4. Team assessment. A team of educators, business personnel, 
graduates, industrial personnel and community people are invited to • 
review and analyze the total vocational program and/or its components. 
The team can make recommendations (or bringing the program and/or its 
components closer to meeting the combined needs of the students, 
industry, businesses and the community. 

Another method of team assessment involves two teams of individuals; 
one to plan the evaluation activity and prepare the materials and the other 
to actually conduct the evaluation. The planning team assembles all 
available information about the program and its components. The 
evaluation team observes, interviews and studies the assembled 
information in an attempt to formulate conclusions about the program. The 
evaluation team then provides consultant assistance and suggests ways 
to improve the identified deficiencies. 

5. The Audit. The purpose of the audit procedure is to report - achieveo 
outcomes and to place these side-by-side with intended outcomes to 
determine the degree of program success. This purpose is most easily 
accomplished through the use of audit forms. Audit forms can be 
developed locally or purchased. The purchased forms are considered 
samples because program ojectives vary to some extent from program to 
program. 

An audit can be either internal or external. An internal audit is performed 
by personnel within the program: students, teachers, and administrators. 
An external audit is performed by personnel outside the program: 
graduates, neighboring school personnel, business and industrial 
representatives and community people. An audit can be applied to the 
total program, a vocational service area, or an individual program, course 
or activity. 

6. Cost analysis evaluation. A system of evaluation is not complete without 
a means of assessing costs in relation to program attributes and 
outcomes. Cost studies attempt to correlate program costs to program 
outcomes, thereby facilitating program decision making. Two programs 
or courses having similiar goals and objectives but different costs should 
be examined to determine if the additional expenditure is necessary for 
one course as opposed to the other. 

D. Who Should Be Involved in the Evaluation? After reviewing the possible 
approaches to evaluation, it can be determined that it is not possible for one 
person to be responsible for accomplishing all of the evaluation requirements 
and related tasks. It is important to involve a number of people within and 
without the program in the evaluation. The scope of the evaluation will 
determine who is to be involved. As a rule, all individuals who share an 
interest in the course, program or activity being evaluated should be involved 
to ihe degree corresponding to their interest. People who have participated 
in the evaluation will be in the best position to implement changes considered 
necessary by the evaluation. 
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1. Administrative personnel. The administrative staff at all levels must be 
involved in all program evaluations. Even if not directly responsible for 
initiating the evaluation activity, the administrative personnels' support 
and sanction are important. In addition, the administration personnel can 
act as advisors to the evaluation process, liason between an outside team 
and inside personnel, insure adequate financial resources, release time if 
necessary, and a monitor of progress. Administration will play an 
important role in how the results are translated into remedies; therefore, it 
Is vital that they be committed and involved. 

2. Instructional personnel. Instructional personnel should be involved in 
evaluating their own programs and activities as well as those of others. 
They should be involved because they are more knowledgeable about 
program and course specifications than any other group. Instructional 
personnel should play a key role in the development of the evaluation 
goals as well as how the evaluation is to be conducted and the results 
interpreted. Their involvement will also help reduce apprehension 
concerning the possible misuse of the results. 

3. Students. For many evaluation activities, such as those concerning 
instructional personnel, support personnel and course and program 
operation, students are in the best position to observe the every day 
activities that an outside evaluator is likely to miss. 1 herefore, student 
involvement in the preparation and administration of the evaluation is 
critical. 

4. Support personnel. The opinion of support personnel, counselors, 
placement personnel, secretaries, custodians, etc. have a potential for 
contributing to the evaluation regardless of their chain of command level. 
These individuals will many times possess the expertise and information 
necessary for a successful evaluation. 




5. Advisory committee members and other citizens. Advisory committee 
members have already made a committment to vocational education and 
therefore can encourage other local personnel to cooperate in the 
evaluation activities. Their expertise can be utilized in making 
recommendations for improvement after the evaluation as well as 
planning for evaluation. 

i J 
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Other citizens you may wish to involve in the evaluation are members of 
labor organizations, industrial councils, parent groups, Chamber of 
Commerce, employment services, etc. These people can provide 
information from their perspectives about crucial parts of the program, as 
they are "users" of the "products" of vocational education. 

6. External experts. It is often beneficial or even necessary to involve 
evaluation experts from private consulting firms, universities, and other 
education and training organizations or agencies. External experts can be 
used to help in the design and implementation of the evaluation system, 
identification of evaluation needs, use of evaluation results or as directors 
of the evaluation. The actual component to be evaluated and the 
technique chosen will dictate which individuals should be chosen to 
participate in a particular evaluation activity. 

Procedure for Designing an Evaluation System 

Before specific procedures for conducting an evaluation are determined, there is a 
need to develop an overall structure within which these activities can and should be 
conducted. Several procedures which should be followed in designing an overall 
evaluation system prior to its initiation follow. 

A. Focusing the evaluation. Spell out the objective of the evaluation and define 
the policies which will govern the process of evaluation. The audience of the 
evaluation results should be identified prior to any activity or collection of 
information. Decision levels should be identified; whether administrative level, 
instructional level or budgetary level, and then specific decisions to aid in 
structuring the evaluation can be made. The next step in focusing the 
evaluation system should be formulation of evaluation questions. The 
evaluation questions are primarily the rewriting of decision situations. 

B. Selecting appropriate evaluation activities. After the decision pituations and 
levels are identified and the evaluation questions definitely delineated, 
activities can be selected that enable program personnel to obtain the 
informtion necessary to make decisions. Once the data or information is 
identified, those considered the most important should be given pric rity in 
terms of selection of evaluation activities. 

C. Structuring Evaluation Activities. Each person participating in these activities 
should be involved in all discussion relating to the study. This will give them 
an increased sense of in volvement which contributes greatly to their 
commitment to the activity. 

1. Determine who should be involved. This ranges from program 
personnel-administrators, instructors, and support personnel through 
students, community representation and external experts. 

2. Identify activity tasks. Tasks should be described and explained in 
complete detail. It is imperative that the evaluation personnel know 
exactly what to do to insure that they can handle unexpected problems 
which have been unrehearsed. 
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3. Inform personnel. If all members of Ihe evaluation team are properly 
informed, they will be in a position to help one another, adequately explain 
the evaluation goals to others, make good decisions and comprehend the 
problems that are met by other team members. 

^. Assign personnel responsibilities. The scope and extensiveness of the 
evaluation activity will determine the number of individuals required. The 
evaluation director assumes overall leadership. A leader should be 
identified for each task. If no special knowledge is required, the director 
may ask for volunteers. When expertise is required, the director may 
select individuals he feels are qualified to do the job. 

5. Sequence the tasks. To initiate the sequencing of tasks, list all the tasks 
and Identify those which are dependent upon another task. Put the tasks 
in the order in which they must be completed. Many activities can be 
conducted simultaneously. Place the activities in a pictorial schedule. 
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6. Determine time estimates. After the activities have been sequenced and 
scheduled, time estimates need to be attached to each task. A sequence 
schedule should be prepared showing each activity, estimated completion 
date and the name of the leader or person responsible for the completion 
of the activity. 



Final Report Outline Example 

I. Abstract 

Occupies one or two pages; identifies the school; and concisely and simply 
summarizes the objectives, findings, conclusions, and recommendations. 

II. Introductory Section 

A. Introduction: Vocational Program 

This section should give a description of the institution's vocational 
program. It may include statements on the institution's vocational 
education philosophy and objectives. 

B. Objectives of Local School or College Evaluation 

This section should include a clear, concise listing of the objectives 
established within the framework of the evaluation. These objectives give 
direction to the institution's evaluation effort. 



C. Description of Activities 

In this section, a detailed description should be given of each activity 
included in the evaluation. A time schedule and other materials may also 
be included. 

D. Staff Involvement 

This section should present information focusing upon the extent and type 
of staff involvement. It should address the outcomes, attitudes, type of 
committees, functions, philosophy changes, successes, failures, and so 
on. 



E. Citizen Involvement 

Facts about citizen involvement in the evaluation are presented in this 
section. It reports types, size and representation of committees, 
functions, attitudes, outcomes, successes failures, and so on. 

F. Consultant Involvement in Project 

This section includes information focusing upon the role of any 
consultants in the evaluation. 

G. Public Relations 

This section deals with the type, method, and scope of public relations 
used in connection with the evaluation. 



III. Findings Section 

A. Committee Activities 
This section reports the number and type of committees and the scope of 
activities of each. 

B. Committee Recommendations 
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The recommendations of each committee are presented in this section. 
Each recommendation should be accompanied by supportive data and the 
other information. 

C. Follow-up Studies 

If employer surveys and follow-up studies of graduates and dropouts were 
made, the results and generalizations are reported in this section. 

D. Curricular Studies 

If studies of present or potential curricula were made, the results are 
reported in this section. 

E. Other Types of Studies 

If occupational surveys, attitude studies, or other types of studies were 
made, the results are reported in this section. 

F. Changes Resulting from or Related to Local Program Evaluation 

Any changes, innovations, adoptions, or plans already made are reported 
in this section. 

IIV. Conclusions and Recommendations 

A. Conclusions 

The conclusions should be organized according to the objectives that 
were drawn up for the evaluation. 

B. Recommendations 

All recommendations and future plans for evaluation should be reported in 
this section. 

V. Appendix 

A. Questionnaires 

B. Checklists 

C. Committee membership lists 

D. Public relations materials 

1 . Determine format for reporting data. Depending on the sources of 
information, the reporting format could include tallies, percentages, 
histograms or bar graphs and averages. 

2. Determine format for reporting judgements. The formats for reporting 
judgements are many. One that has proved its usefulness consists of 
three columns: one for conclusions, one for recommendations and 
the third for suggested solutions. Format for Reporting Judgements 



Conclusions 


Recommendations 


Sugguested Solutions 
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E. Developing Management Evaluation Objectives. 

Management evaluation objectives are statements of intent designed to 
communicate the activities of the evaluation process to all the people 
involved. Management evaluat ; on objectives should be prepared for each 
task within an evaluation activity. 

F. Incorporating a Means of Meta Evaluation. 

Meta evaluation is an evaluation of the evaluation system. It is very 
important that those involved with or affected by an evaluation monitor the 
purposes, processes and outcomes of the evaluation system during its 
development and implementation stages. 

III. Using the Evaluation Data. 

The collection and analysis of data should not be the end product. The data 
collected should be used as pari of the decision making process. Listed 
below are six suggestions to follow if the evaluation results are to influence 
the decision making process. 

1 . Involve as many different groups as possible in the evaluation 
process. The results of the evaluation are more likely to be used if a 
great nember of people know about it. 

2. Avoid involvement in an evaluation unless assured the results are to 
be used. Without this assurance, the evaluation becomes no more 
than an exercise. 

3. Build decision choices into the evaluation plan. The results are more 
likely to be used if the evaluation plan includes several outcome 
possibilities and prescribes a course of action for each. 

4. Be sensitive to the relevant, interna! politics of the area being 
evaluated. Evaluation data alone will not produce change. Groups 
and individuals interpret the data and allow it to influence their 
decision making. The evaluation director must be aware of the 
political realities of the area being evaluated. 

5. Make strong operational recommendations in the final report. The 
final report could well be the only part of the evaluation that gets 
reproduced, circulated and read. 

6. Give a copy of the results to the administration, the advisory 
committee, the board of education and the media. Recommendations 
are more likely to be listened to if they are communicated clearly to 
the people responsible for decision making. 

V. How to Avoid Common Evaluation Pitfalls. 

Mistakes are hard to cover. Most mistakes are a function of either 
expectations or schedules. The evaluator cannot anticipate every problem 
that will arise and plan for it. The evaluator must make everyone involved 
aware of his expectations or the general way in which the evaluation is to be 
carried out. Some suggestions: 

1. Lay out a detailed schedule of the evaluation plan. 

2. Prepare a list of responsibilities for each person involved in the evaluation. 

3. Conduct periodic briefings for all those involved in the evaluation. 
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4. Do not assume anything. 

5. Do not bo afraid to alter things as you go along, if it is necessary. 

6. Do not avoid statistics; seek help if necessary. 

7. If there is a great amount of data, use a computer to analyze them. 

8. Draw conclusions justified only by the data. 

9. Do not let anyone other than the evaluation director write the final report. 

The Missouri School Improvement Program (MSIP) 

Effective school improvement programs involve as many people as possible in both the 
initial planning for the program as well as the implementation of the program. Several 
Missouri school districts already have effective school improvement and planning 
processes in place. It is not the intent of the Missouri School Improvement Program to 
require a specific model for school improvement, but to allow districts enough flexibility 
to respond in ways that best fit their individual needs and resources. A review of 
literature concerning school improvement can help districts identify the components of 
effective school improvement programs. There are a variety of strategies for 
developing and implementing successful school improvement plans that will directly 
enhance the educational opportunities for students. Districts are encouraged to 
develop those strategies that they feel will work best for them. 

Since districts need to submit a plan to the state's School Improvement Committee, the 
initial plan will, of nacessity, be general in nature and reflect the district's commitment 
to improvement and its priorities for addressing the identified concerns. As efforts 
toward school progress within the districts, more detailed plans, building level plans and 
other plans, can be developed to help address the areas of concern and achieve those 
goals which were identi'ied. The State Supervisor of Instruction and the appropriate? 
program specialist will monitor the district's progress toward developing and imple- 
menting the district's school improvement plan. 

School districts will use these school improvement plans to evaluate their educational 
processes and to make plans for district-wide improvements. In the assessment 
process, a team of educators will validate the efforts a district has made to achieve the 
objectives which are embodied in the district's school improvement plan. 

The school improvement review team's work will be facilitated by the districts' 
assembling their documents and materials which confirm or validate their efforts to 
meet the process and performance standards and by placing this material in documen- 
tation files. 

An outline of some of the materials and resources which the team will need to review 
is given below. 



Curriculum and Instruction 

1 . Curriculum guides for all instructional programs. 

2. Individual course descriptions. 

3. Any articulated objectives or plans used to define the curriculum. 

4. All board policies on curriculum and instruction. 

5. District's philosophy/goals/mission statement. 

6. District's curriculum committee meeting and activities. 
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Assessment 

1 . Board policies regarding assessment. 

2. Board minutes of its annual review of assessment. 

3. Assessment plan and testing calendar. 



Chapter I 



1. 


Financial records 


2. 


Parent participation policy 


3. 


Complaint procedure 


4. 


Master Eligibility List 


5. 


Individual class schedules and rosters 


6. 


Sustained effect study 


7. 


Proper credentials of Chapter I staff 


8. 


Attendance center selection and participation information 



Vocational Education 

1 . Vocational education materials and documents for each program area to 
include: 

a. Statement of philosophy for the department or occupational program area. 

b. Statement of program objectives. 

c. Class schedule. 

d. Membership roster of the advisory committee. 

e. List of advisory committee activities. 

f . Instructional objectives and course outlines. 

g. Development plans and long-range needs. 

h. Current enrollment information. 

i. Placement and follow-up data, 
j. Employer follow- jp forms. 

k. In-service sched Jles with special population topics identified. 
I. Vocational instructional management system core competency list for 
vocational programs. 



Climate 

1 . Board policies on discipline and conduct. 

2. Sections of student and faculty handbooks concerning discipline, cone :ot, 
attendance, suspension and expulsion. 



Learning Resources Centers 

1. LRC service plan and collection policies. 

2. LRC curriculum. 
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Guidance and Counseling 

1 . Comprehensive guidance plan. 

2. Guidance curriculum. 

Professional Development 

1 . Professional development policies and procedures. 

a. Professional development committee policy. 

b. Mentoring policy. 

c. Professional development requirements. 

d. Reimbursement policies and procedures. 

Supplemental Rograms 

1 . Board policies regarding - 

a. Extra-curricular and co-curricular activities. 

b. Fund raising activities. 

c. Use of school facilities by community groups. 

2. Chapter II, Title II drug-free schools and community programs. 

Governance and Administration 

1 . Board policies and procedures. 

2. Board minutes and agendas. 

3. Job description for employers. 

4. Compensation plan/salary schedules. 

5. Informational brochures and/or materials. 

6. Building committee information. 

Facilities 

1. Long-range plans for maintenance and/or renovation 

2. Schedules of maintenance/renovations currently in program 

Safety 

1. Safety information form to be completed. 

School Health Service 

1 . Comprehensive health services plan. 

2. Health curriculum. 

3. Location of cumulative health records, emergency medical records and 
immunization records. 

4. Board policies on health services, communicable diseases, child abuse and 
neglect, and drug-free environment. 
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Nutrition Services 

1 . Free/reduced price mea! procedure and criteria. 

2. Number or percentages of students participating in lunch program and 
receiving free/reduced price meals. 

3. Written nutrition plan. 



Transportation 

1 . Inspection report for district buses. 

2. Proof of liability or property damage insurance on buses. 

3. Transportation contracts with outside provider. 

4. State transportation report. 

5. Evidence of transportation services emergency and safety training. 



Performance 

1 . Student achievement data. 

2. Student follow-up data. 

3. Drop-out/persistence to graduation data. 

4. Optional performance standards. 



Procedures for State's Review of District's School Improvement 
Programs 

Missouri's school districts are scheduled for review on a five-year cycle. 

Each district scheduled for review will identify a contact person <rom the district to be 
responsible for coordinating the review effort. The on-site review team leader or State 
Supervisor will work out details in regard to developing the on-site interviews, schedule , 
administering the advance questions.and making other logistical arrangements. 

Approximately eight weekspriortotheon-site review, advance questions will be mailed 
to the district. These questions give the teachers, administrators, support staff and 
students an opportunity to assist in the assessment of the school district. The 
perceptions of those who work and learn in the district are an important part of the 
assessment process. 

Advanced questionnaires are provided for all professional staff, central office admin- 
istrators, building principals, classroom teachers, librarians, counselors, board of 
education members, nutrition services staff, transportation staff, maintenance staff, 
clerical staff and instructional aides. Responses can be separated by building level! 
teaching experience, education level, subject area and other ways for better 
interpretation of the results. 

The student questionnaires are self-contained, scan-sensitive answer sheets which 
means the students can mark their responses on the questionnaire. There are two 
forms of the student questionnaire. The elementary form (grades 5 through 8), and the 
secondary form (grade 7 through 12). The questionnaires themselves are dem • 
graphic and opinion questions for the student. The student has the option not to 
^ respond to any item. 
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Advance staff- and student-completed scan forms will be returned to the Supervisor of 
Instruction section for processing and analyzing the advance data as well as sharing 
this data with team members prior to the on-site visit. 



The On-site Review Team 

The on-site review team will be composed of approximately one-third state department 
staff members, one-third practicing educators from across the state, and one-third 
members from college and university staffs, school board members from other 
dis tricts, civic leaders from other communities, and representatives from business and 
industry. All team leaders will have completed a two-day training session and team 
members participate in a one-day training session. 



Team Orientation 

Before the review begins, the r- Wew team leader and the State Supervisor of 
Instruction will meet with the on-s : '.o review team. At this meeting, they will discuss the 
results of the advance questionnaire, the information provided by the district, the 
resource information and review procedures to be followed during the on-site review. 
This is usually accomplished the night before the on-site visits, along with the district 
orientation. 



On-site Interview Procedures 

The State Supervisor of Instruction will prepare a list of staff members to be interviewed 
in each district. This list will contain a random selection of teachers representing all 
grade levels and all subject matter areas. Basically, all counselors, librarians, and 
administrators will be interviewed. Support staff will be interviewed to the extent 
possible. 

The State Supervisor of Instruction will work with the district contact person in 
developing an interview schedule. Each interviewer will be allowed time to see the 
facilities and walk through some of the classrooms during classes. The review team 
should not intrude, interrupt or inconvenience the district any more than necessary. 

On-site Review Schedule 

The size of the review team and the length of time spent in the district will be determined 
by the size of the district, programs and services offered, student/teacher population 
and the number of buildings in the district. The format most frequently used is for the 
team to arrive the evening prior to the on-site interview and conduct a team orientation 
and a district orientation that evening. The first morning of the on-site review the 
building administrators orient the respective team members to their buildings Then 
interviews are conducted for most of the first day. Interviews may extend into the 
second day. The second day affords team members the opportunity to pick up 
additional information and to further review the documentation file. When the individual 
area teams have reached a consensus on all issues and concerns, the report writina 
Jrocess begins. y 
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The Exit Conference 

The review team leader, the State Supervisor, and perhaps a few individual area team 
leaders will hold an exit conference with the district administration. At the exit 
conference, the entire report is not reviewed; only the major areas of strength and 
concerns are discussed. If there are any major disagreements or concerns, further 
information may be obtained. The district will receive the full assessement at a later 
date. 

District assessment is an important component of the Missouri School Improvement 
Program because it gives staff members and team members the opportunity to make 
suggestions for improvement of the student's educational program. 



Summary and Review 

Evaluation is an integral part of any decision-making process. School districts have 
increasingly been called upon to qualify their existing programs or to choose those on- 
going programs that they will eliminate in order to meet budget restrictions. In this 
context, t:ie importance of the evaluation and the evaluator role in the general scheme 
of decision making has increased. 

All personnel have a "need to know" when it comes to evaluation of instructional 
programs. Evaluation should not be an esoteric collection of jargon, statistics, and 
techniques understood only by a specialized few. Everyone involved and everyone 
that has a stake in the program must understand how the evaluation is or was 
accomplished. 

The purpose of this module has been to expose the student to various methods of 
evaluation that can be used for regular and cooperative vocational programs. While 
not designed to give the teacher skill in planning and conducting regular and cooperative 
vocationa! program evaluations, every teacher will help with the evaluation process and 
needs to be prepared to utilize pertinent findings and recommendations to improve his/ 
her program. 
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1 . Name the five reasons why a vocational education program should be evaluated periodically, 
a. 



b. 



d. 



2. Name those things that should be evaluated in a vocational education program. 



c. 



d. 
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3. What six techniques can be used to evaluate a vocational education program? 
a. 



b. 



c. 



d. 



e. 



f. 
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4. Who should be involved in the evaluation of a vocational education program and why? 



5. What are the procedures to be followed in designing an evaluation system? 



6. How should the data collected during an evaluation be used? 
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Question #1 Answer 

A. As a planning aid 

B. As a decision making aid 

C. As a program personnel aid 

D. As a student program aid 

Question #2 Answer 

A. Decisions made by educators 

B. Planning 

C. Programming 

D. Implementing 

E. Restructuring 

Question #3 Answer 

A. Student assessment 

B. Graduate assessment 

C. Employer assessment 

D. Team assessment 

E. Audit 

F. Cost analysis assessment 
Question #4 Answer 

Administrative personnel; instructional personnel; students; support personnel; advisory committee members 
and other citizens; and external experts. 

Question #5 Answer 

Focusing the evaluation 

Selecting appropriate evaluation activities 
Structuring evaluation activities 
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Determine who should be involved 

Identify activity tasks 

Assign personnel responsibilities 

Sequence tasks 

Determine time estimates 
Structure the evaluation outcome 

Determine format for reporting data 

Determine format for reporting judgements 
Develop management evaluation objectives 
Incorporate a means of meta evaluation 

Evaluate the evaluation system 

Question #6 Answer 

Make a copy of evaluation results available to the administrators, the advisors committee, the Board of 
Education, and the local media. 
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Develop a written plan for an evaluation ol the program area in which you teach. 



3. 
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Evaluation Plan Checklist 

Place a check mark in the correct box opposite the checklist question. 

Your Plan: Yes No 

1 . Provide for the following persons to be involved in the evaluation: 

a. Administrative personnel ( ) ( ) 

b. Instructional personnel ( ) ( ) 

c. Students ( ) ( ) 

d. Support personnel O ( ) 

e. Advisory committee O ( ) 

f. External experts ( ) ( ) 

g. Others ( ) f\ 

2. Include a description of the area served ( ) ( ) 

3. Include a statement of philosophy ( ) ( ) 

4. Include a plan of evaluation objectives O ( ) 

5. Describe the specific responsibilities assigned to the evaluation 

director / \ ( \ 

6. List the major activities to be completed in sequence ( ) ( ) 

7. Provide a time estimate for each activity to be completed ( ) ( ) 

8. Include a list of items and the amount of money which will need to 

be budgeted / \ / \ 

9. Describe the format for reporting dates ( ) ( ) 

1 0. Describe the method that will be used for dissemination of final 

report / \ / \ 

1 1 . Describe the strategies to be used for implementation of 
recommendations advanced in final report 



